Multimodal pre-screening can predict BCI performance variability: A novel subject-specific experimental scheme.

BACKGROUND: Brain-computer interface (BCI) systems currently lack the required robustness for long-term daily use due to inter- and intra-subject performance variability. In this study, we propose a novel personalized scheme for a multimodal BCI system, primarily using functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), to identify, predict, and compensate for factors affecting competence-related and interfering factors associated with performance. METHOD: 11 (out of 13 recruited) participants, including five participants with motor deficits, completed four sessions on average. During the training sessions, the subjects performed a short pre-screening phase, followed by three variations of a novel visou-mental (VM) protocol. Features extracted from the pre-screening phase were used to construct predictive platforms using stepwise multivariate linear regression (MLR) models. In the test sessions, we employed a task-correction phase where our predictive models were used to predict the ideal task variation to maximize performance, followed by an interference-correction phase. We then investigated the associations between predicted and actual performances and evaluated the outcome of correction strategies. RESULT: The predictive models resulted in respective adjusted R-squared values of 0.942, 0.724, and 0.939 for the first, second, and third variation of the task, respectively. The statistical analyses showed significant associations between the performances predicted by predictive models and the actual performances for the first two task variations, with rhos of 0.7289 (p-value = 0.011) and 0.6970 (p-value = 0.017), respectively. For 81.82 % of the subjects, the task/workload correction stage correctly determined which task variation provided the highest accuracy, with an average performance gain of 5.18 % when applying the correction strategies. CONCLUSION: Our proposed method can lead to an integrated multimodal predictive framework to compensate for BCI performance variability, particularly, for people with severe motor deficits.

A dynamical graph-based feature extraction approach to enhance mental task classification in brain-computer interfaces.

Graph theoretic approaches in analyzing spatiotemporal dynamics of brain activities are under-studied but could be very promising directions in developing effective brain-computer interfaces (BCIs). Many existing BCI systems use electroencephalogram (EEG) signals to record and decode human neural activities noninvasively. Often, however, the features extracted from the EEG signals ignore the topological information hidden in the EEG temporal dynamics. Moreover, existing graph theoretic approaches are mostly used to reveal the topological patterns of brain functional networks based on synchronization between signals from distinctive spatial regions, instead of interdependence between states at different timestamps. In this study, we present a robust fold-wise hyperparameter optimization framework utilizing a series of conventional graph-based measurements combined with spectral graph features and investigate its discriminative performance on classification of a designed mental task in 6 participants with amyotrophic lateral sclerosis (ALS). Across all of our participants, we reached an average accuracy of 71.1%±4.5% for mental task classification by combining the global graph-based measurements and the spectral graph features, higher than the conventional non-graph based feature performance (67.1%±7.5%). Compared to using either one of the graphic features (66.3%±6.5% for the eigenvalues and 65.9%±5.2% for the global graph features), our feature combination strategy shows considerable improvement in both accuracy and robustness performance. Our results indicate the feasibility and advantage of the presented fold-wise optimization framework utilizing graph-based features in BCI systems targeted at end-users.

A Graph-Based Feature Extraction Algorithm Towards a Robust Data Fusion Framework for Brain-Computer Interfaces.

OBJECTIVE: The topological information hidden in the EEG spectral dynamics is often ignored in the majority of the existing brain-computer interface (BCI) systems. Moreover, a systematic multimodal fusion of EEG with other informative brain signals such as functional near-infrared spectroscopy (fNIRS) towards enhancing the performance of the BCI systems is not fully investigated. In this study, we present a robust EEG-fNIRS data fusion framework utilizing a series of graph-based EEG features to investigate their performance on a motor imaginary (MI) classification task. METHOD: We first extract the amplitude and phase sequences of users' multi-channel EEG signals based on the complex Morlet wavelet time-frequency maps, and then convert them into an undirected graph to extract EEG topological features. The graph-based features from EEG are then selected by a thresholding method and fused with the temporal features from fNIRS signals after each being selected by the least absolute shrinkage and selection operator (LASSO) algorithm. The fused features were then classified as MI task vs. baseline by a linear support vector machine (SVM) classifier. RESULTS: The time-frequency graphs of EEG signals improved the MI classification accuracy by ∼5% compared to the graphs built on the band-pass filtered temporal EEG signals. Our proposed graph-based method also showed comparable performance to the classical EEG features based on power spectral density (PSD), however with a much smaller standard deviation, showing its robustness for potential use in a practical BCI system. Our fusion analysis revealed a considerable improvement of ∼17% as opposed to the highest average accuracy of EEG only and ∼3% compared with the highest fNIRS only accuracy demonstrating an enhanced performance when modality fusion is used relative to single modal outcomes. SIGNIFICANCE: Our findings indicate the potential use of the proposed data fusion framework utilizing the graph-based features in the hybrid BCI systems by making the motor imaginary inference more accurate and more robust.

Graph-based Recurrence Quantification Analysis of EEG Spectral Dynamics for Motor Imagery-based BCIs.

Despite continuous research, communication approaches based on brain-computer interfaces (BCIs) are not yet an efficient and reliable means that severely disabled patients can rely on. To date, most motor imagery (MI)-based BCI systems use conventional spectral analysis methods to extract discriminative features and classify the associated electroencephalogram (EEG)-based sensorimotor rhythms (SMR) dynamics that results in relatively low performance. In this study, we investigated the feasibility of using recurrence quantification analysis (RQA) and complex network theory graph-based feature extraction methods as a novel way to improve MI-BCIs performance. Rooted in chaos theory, these features explore the nonlinear dynamics underlying the MI neural responses as a new informative dimension in classifying MI. METHOD: EEG time series recorded from six healthy participants performing MI-Rest tasks were projected into multidimensional phase space trajectories in order to construct the corresponding recurrence plots (RPs). Eight nonlinear graph-based RQA features were extracted from the RPs then compared to the classical spectral features through a 5-fold nested cross-validation procedure for parameter optimization using a linear support vector machine (SVM) classifier. RESULTS: Nonlinear graph-based RQA features were able to improve the average performance of MI-BCI by 5.8% as compared to the classical features. SIGNIFICANCE: These findings suggest that RQA and complex network analysis could represent new informative dimensions for nonlinear characteristics of EEG signals in order to enhance the MI-BCI performance.

Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework.

Multimodal data fusion is one of the current primary neuroimaging research directions to overcome the fundamental limitations of individual modalities by exploiting complementary information from different modalities. Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) are especially compelling modalities due to their potentially complementary features reflecting the electro-hemodynamic characteristics of neural responses. However, the current multimodal studies lack a comprehensive systematic approach to properly merge the complementary features from their multimodal data. Identifying a systematic approach to properly fuse EEG-fNIRS data and exploit their complementary potential is crucial in improving performance. This paper proposes a framework for classifying fused EEG-fNIRS data at the feature level, relying on a mutual information-based feature selection approach with respect to the complementarity between features. The goal is to optimize the complementarity, redundancy and relevance between multimodal features with respect to the class labels as belonging to a pathological condition or healthy control. Nine amyotrophic lateral sclerosis (ALS) patients and nine controls underwent multimodal data recording during a visuo-mental task. Multiple spectral and temporal features were extracted and fed to a feature selection algorithm followed by a classifier, which selected the optimized subset of features through a cross-validation process. The results demonstrated considerably improved hybrid classification performance compared to the individual modalities and compared to conventional classification without feature selection, suggesting a potential efficacy of our proposed framework for wider neuro-clinical applications.

P300 latency jitter and its correlates in people with amyotrophic lateral sclerosis.

OBJECTIVE: People with amyotrophic lateral sclerosis (ALS) can benefit from brain-computer interfaces (BCIs). However, users with ALS may experience significant variations in BCI performance and event-related potential (ERP) characteristics. This study investigated latency jitter and its correlates in ALS. METHODS: Electroencephalographic (EEG) responses were recorded from six people with ALS and nine neurotypical controls. ERP amplitudes and latencies were extracted. Classifier-based latency estimation was used to calculate latency jitter. ERP components and latency jitter were compared between groups using Wilcoxon rank-sum tests. Correlations between latency jitter and each of the clinical measures, ERP features, and performance measures were investigated using Spearman and repeated measures correlations. RESULTS: Latency jitter was significantly increased in participants with ALS and significantly negatively correlated with BCI performance in both ALS and control participants. ERP amplitudes were significantly attenuated in ALS, and significant correlations between ERP features and latency jitter were observed. There was no significant correlation between latency jitter and clinical measures. CONCLUSIONS: Latency jitter is increased in ALS and correlates with both BCI performance and ERP features. SIGNIFICANCE: These results highlight the associations of latency jitter with BCI performance and ERP characteristics and could inform future BCI designs for people with ALS.

Electrical and Hemodynamic Neural Functions in People With ALS: An EEG-fNIRS Resting-State Study.

OBJECTIVE: Amyotrophic lateral sclerosis (ALS) is a complex neurodegenerative disease that causes the progressive loss of voluntary muscle control. Recent studies have reported conflicting results on alterations in resting-state functional brain networks in ALS by adopting unimodal techniques that measure either electrophysiological or vascular-hemodynamic neural functions. However, no study to date has explored simultaneous electrical and vascular-hemodynamic changes in the resting-state brain in ALS. Using complementary multimodal electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) recording and analysis techniques, we explored the underlying multidimensional neural contributions to altered oscillations and functional connectivity in people with ALS. METHODS: 10 ALS patients and 9 age-matched controls underwent multimodal EEG-fNIRS recording in the resting state. Resting-state functional connectivity (RSFC) and power spectra of both modalities in both groups were analyzed and compared statistically. RESULTS: Increased fronto-parietal EEG connectivity in the alpha and beta bands and increased interhemispheric and right intra-hemispheric fNIRS connectivity in the frontal and prefrontal regions were observed in ALS. Frontal, central, and temporal theta and alpha EEG power decreased in ALS, as did parietal and occipital alpha EEG power, while frontal and parietal hemodynamic spectral power increased in ALS. SIGNIFICANCE: These results suggest that electro-vascular disruption in neuronal networks extends to the extra-motor regions in ALS patients, which can ultimately introduce novel neural markers of ALS that can be exploited further as diagnostic and prognostic tools.

Enhancing Communication for People in Late-Stage ALS Using an fNIRS-Based BCI System.

OBJECTIVE: Brain-computer interface (BCI) based communication remains a challenge for people with later-stage amyotrophic lateral sclerosis (ALS) who lose all voluntary muscle control. Although recent studies have demonstrated the feasibility of functional near-infrared spectroscopy (fNIRS) to successfully control BCIs primarily for healthy cohorts, these systems are yet inefficient for people with severe motor disabilities like ALS. In this study, we developed a new fNIRS-based BCI system in concert with a single-trial Visuo-Mental (VM) paradigm to investigate the feasibility of enhanced communication for ALS patients, particularly those in the later stages of the disease. METHODS: In the first part of the study, we recorded data from six ALS patients using our proposed protocol (fNIRS-VM) and compared the results with the conventional electroencephalography (EEG)-based multi-trial P3Speller (P3S). In the second part, we recorded longitudinal data from one patient in the late locked-in state (LIS) who had fully lost eye-gaze control. Using statistical parametric mapping (SPM) and correlation analysis, the optimal channels and hemodynamic features were selected and used in linear discriminant analysis (LDA). RESULTS: Over all the subjects, we obtained an average accuracy of 81.3%±5.7% within comparatively short times (< 4 sec) in the fNIRS-VM protocol relative to an average accuracy of 74.0%±8.9% in the P3S, though not competitive in patients with no substantial visual problems. Our longitudinal analysis showed substantially superior accuracy using the proposed fNIRS-VM protocol (73.2%±2.0%) over the P3S (61.8%±1.5%). SIGNIFICANCE: Our findings indicate the potential efficacy of our proposed system for communication and control for late-stage ALS patients.

Frontal Functional Network Disruption Associated with Amyotrophic Lateral Sclerosis: An fNIRS-Based Minimum Spanning Tree Analysis.

Recent evidence increasingly associates network disruption in brain organization with multiple neurodegenerative diseases, including amyotrophic lateral sclerosis (ALS), a rare terminal disease. However, the comparability of brain network characteristics across different studies remains a challenge for conventional graph theoretical methods. One suggested method to address this issue is minimum spanning tree (MST) analysis, which provides a less biased comparison. Here, we assessed the novel application of MST network analysis to hemodynamic responses recorded by functional near-infrared spectroscopy (fNIRS) neuroimaging modality, during an activity-based paradigm to investigate hypothetical disruptions in frontal functional brain network topology as a marker of the executive dysfunction, one of the most prevalent cognitive deficit reported across ALS studies. We analyzed data recorded from nine participants with ALS and ten age-matched healthy controls by first estimating functional connectivity, using phase-locking value (PLV) analysis, and then constructing the corresponding individual and group MSTs. Our results showed significant between-group differences in several MST topological properties, including leaf fraction, maximum degree, diameter, eccentricity, and degree divergence. We further observed a global shift toward more centralized frontal network organizations in the ALS group, interpreted as a more random or dysregulated network in this cohort. Moreover, the similarity analysis demonstrated marginally significantly increased overlap in the individual MSTs from the control group, implying a reference network with lower topological variation in the healthy cohort. Our nodal analysis characterized the main local hubs in healthy controls as distributed more evenly over the frontal cortex, with slightly higher occurrence in the left prefrontal cortex (PFC), while in the ALS group, the most frequent hubs were asymmetrical, observed primarily in the right prefrontal cortex. Furthermore, it was demonstrated that the global PLV (gPLV) synchronization metric is associated with disease progression, and a few topological properties, including leaf fraction and tree hierarchy, are linked to disease duration. These results suggest that dysregulation, centralization, and asymmetry of the hemodynamic-based frontal functional network during activity are potential neuro-topological markers of ALS pathogenesis. Our findings can possibly support new bedside assessments of the functional status of ALS' brain network and could hypothetically extend to applications in other neurodegenerative diseases.